1. Field of the Invention
This invention relates generally to data analysis and control systems and, more specifically, to a method of providing a universal data measurement, analysis and control system for a variety of types of input data.
2. Description of Related Art
Despite all of the technological advances in software, the software industry has experienced an explosion of cost and schedule overruns in the billions of dollars (see for example GAO 1994 report.sup.1). These overruns continue to focus on the software project management problem of accurately estimating the cost and schedule to complete a project and to accurately monitor these forecasts, as well as status throughout the project life cycle. The general solution has been to use software cost models such as COCOMO or PRICE-S along with a project management tool, such as Microsoft Project. However, these models' performance has been less than adequate, as evidenced by the GAO report. The root cause of the problem can be traced to the data these tools inherently rely on. Since all software costing models must be calibrated to measurements obtained from many projects, each having slightly different measurement protocols, one can easily see that a lack of internally consistent data will give rise to poor forecasts. It follows that if the resulting measurements are of dubious value, then all status estimates and forecasts are also questionable. In fact, this same reasoning extends beyond cost and schedule, and the metrics for quality, safety, complexity, and size are also in question. FNT .sup.1 "Improving Mission Performance Through Strategic Information Management and Technology," GAO/AIMD-94-115, May 1994.
The problem is complicated because this data is obtained from a measurement process that is not precisely repeatable; it is primarily an inconsistent manual process. In other words, the problem is that a measurement protocol has not been adequately defined. A measurement protocol relies on, and requires, consistently well-defined measurements and relationships between these measurements. This is generally referred to as a data measurement model or schema. Present metric software tools do not provide this data schema.
Attempts by software engineering tool manufacturers to provide some measurement information have been inconsistently implemented. The primary reason is that the tool manufacturers are solving other problems, such as providing design aids or requirements traceability. Providing proper measurement data is typically an afterthought. In addition, good forecasting techniques rely on frequent and timely measurements maintained consistently over time.
A further complication is that project level forecasts frequently require data obtained from different software tools. If each software tool generates its own measurements, time and effort are required to integrate this data and generate meaningful reports. This analysis is a time consuming process which typically involves a spreadsheet and manual input by a user. In fact, for organizations which have tried this approach, approximately 5% or more of the total project cost went into software measurement collection and reporting, for minimum core level metrics, such as cost, schedule, size and quality.
Most existing software applications provide metrics within the scope of the functionality provided by the software application. An example of this is "Microsoft Project," or "Project." "Project" provides a project management tool which creates a work breakdown structure to organize tasks that must be accomplished for a project. "Project" also provides the capability to measure effort and schedule metrics which are derived from the data stored within the "Project" database. These metrics are limited to the view of data provided by the "Project" package. A mechanism is not provided for understanding the cause or reason for the value of a given metric. In addition, "Project," as well as other tools today, do not provide a continuous historical repository. The result is that trends are difficult to extract, let alone examine and analyze.
Another popular metric tool is the "Amadeus" Measurement System. "Amadeus" provides effort, schedule, defects, and size metrics. "Amadeus" also provides a file system monitor for the automated collection of metrics. The "Amadeus" metric collection tools are limited to source code counters. "Amadeus" provides source code counters for Ada, C++, Csh, and C. "Amadeus" does not provide; however, a mechanism for user-defined metrics, nor for creating and managing integrated data models between applications. This limits analysis to only one activity, the writing and editing of software code. What is really needed is a mechanism for understanding the cause of a metric value.
A consulting firm, "Software Productivity Research," provides a software metric tool which has a historical database to be used for estimation purposes. "Software Productivity Research" does not publish the environments from which the data was gathered, nor do they provide a definition of the data gathered. This leads to a lack of understanding of the measurement protocol. End users must rely on estimations which may or may not correctly reflect their project environment. A system that permits end-users to analyze the measurement protocol by publishing the data gathering environments and data definitions would be very helpful to the process management function.
Another invention, Khoyi, et. al., U.S. Pat. No. 5,206,951, addresses the problem of integration of applications and data from different software applications. Khoyi provides a group of object management data structures and packages of generic routines for performing operating system-type tasks. The object managers perform the object management functions on the data. A table is provided to identify the appropriate object manager to use for a given object. Each object manager supports different exchange formats for data exchange. Khoyi addresses the problem of data exchange but not the gathering of consistent metric data. A system that manages the exchange of data between different software applications and results in consistent metric data is needed.
Tanifuji, et. al., U.S. Pat. No. 4,740,886, solves the problem of modifying code when new mathematical models are introduced into a system. These mathematical models describe the control objects of the system. The usual strategy is to translate a mathematical model to programming code to apply the mathematical model to the real system. When the mathematical model changes, the programming code must be changed and updated. Tanifuji allows the user to enter a mathematical formula into the system then the system automatically translates the formula to a kind of program. This translation is implemented by having the user enter the mathematical formula in a specific format. This format is then converted to mathematical code. Tanifuji does not teach support changes to the control object. It is limited to the current state of the control object. Modifications must be made when the control object is changed. A system that supports the changes of a control object over time as well as changes to the measurement data associated with the control object is needed.
Natarajan, U.S. Pat. No. 4,887,206, provides an interactive system which takes information from engineering change notices released for an item and estimates the impact on the inventory system. Natarajan requires a user to enter the change notice. Once the change notice is entered, the program provides the cost for those items impacted by the change, those items impacted by the change that are in progress, and those items impacted by the change that are in demand. Natarajan accesses an inventory database for all raw material and finished product in which the item is used; a financial database is accessed for cost information. Natarajan does not address the causes of an engineering change proposal, it does not provide an historical repository, and does not provide an automated means for gathering the engineering change proposal. A method for gathering the engineering change proposal data, for determining the causes for the engineering change proposal, and also providing a historical repository would be very beneficial.
A further problem with the prior art is that nowhere is there a comprehensive estimating and forecasting tool which provides data in a consistent and timely manner. As a result of this, in-depth data analysis is cost-prohibitive, and therefore virtually impossible. What is needed is an analysis system that defines a consistent and repeatable measurement model. This system should define a model for maintaining the measured data, a consistent method for harvesting data from within the software development environment, a reporting mechanism for displaying the measurement data in a timely format, and also provide a mechanism for analysis of software process improvement, in addition to software project analysis.